-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
release-20.2: opt: generate lookup joins on partial indexes #54362
Merged
mgartner
merged 3 commits into
cockroachdb:release-20.2
from
mgartner:backport20.2-54157-54217-54281
Sep 16, 2020
Merged
release-20.2: opt: generate lookup joins on partial indexes #54362
mgartner
merged 3 commits into
cockroachdb:release-20.2
from
mgartner:backport20.2-54157-54217-54281
Sep 16, 2020
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mgartner
changed the title
release-20.2: TODO
release-20.2: opt: generate lookup joins on partial indexes
Sep 14, 2020
This was referenced Sep 14, 2020
rytaft
approved these changes
Sep 14, 2020
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
assuming it's approved in #53662
Reviewed 4 of 4 files at r1, 3 of 3 files at r2, 5 of 5 files at r3.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @RaduBerinde)
Oops... #53662 |
RaduBerinde
approved these changes
Sep 15, 2020
Lookup joins are now generated when the ON filters of a join or the right-side Select filters imply the partial index predicate. In several cases, the filters are reduced once implication is proven. This currently prevents optimal plans from being generated in several cases, either because unnecessary joins remaining in the expression tree, or because lookup semi- and anti-joins that should be created are not. I've left TODOs to denote these cases. Release note (performance improvement): The optimizer can now use partial indexes for lookup join operations. This results in potentially more efficient query plans for joins on tables with partial indexes.
This commit updates `GenerateLookupJoins` to generate lookup semi- and anti-joins in cases where the columns in the original ON filters are not covered by the index, but the columns in the filters remaining after proving filter-predicate implication are covered by the index. This is a special case only possible for semi- and anti-joins because they never include columns from the right side of the join in their output columns. See the comment added in `GenerateLookupJoins` for more details. Release note (performance improvement): Lookup semi- and anti-joins are now explored by the optimizer in more cases when the ON filter implies a partial index predicate. This may lead to more efficient query plans in some cases.
Previously, when generating a special case of lookup semi/anti-joins on partial indexes, all columns from the right side of the input join would be included in the lookup join's columns. These columns were included even though they were no longer needed (in this special case, the ON filters have been reduced during filter/predicate implication). Fetching unnecessary columns was was causing a slight difference between the estimated cost of a lookup semi/anti-join on a non-partial index versus a partial index. This is because the coster adds a slight overhead for each column looked-up by the lookup join. This commit correctly assigns the lookup join columns in this special case so that unnecessary columns are not included. In addition, an assertion has been added to `check_expr.go` (run during `make testrace`) to catch future mistakes like this. Some tests that manually build expressions had to be edited to avoid violating this assertion. Release note: None
mgartner
force-pushed
the
backport20.2-54157-54217-54281
branch
from
September 16, 2020 20:29
0bd5664
to
2470946
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport:
Please see individual PRs for details.
/cc @cockroachdb/release